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Abstract 

Background: Standardized Patients (SPs) are actors trained to portray health care patients during the training and 
assessment of health care providers. This paper describes the methods and costs associated with using SPs to 
evaluate the skills of telephone counselors working on a clinical trial that evaluated a telephone smoking cessation 
program tailored for smokers using Department of Veterans Affairs mental health clinics. 

Findings: Conducting the SP exercises required five main steps: (1) Write a SP case description detailing patient 
demographics, demeanor, clinical symptoms and history, and instructions on how to respond to counseling, (2) 
Identify, select and train actors to portray the SP cases; (3) Conduct audio-taped counseling encounters between 
the SPs and counselors, (4) Rate the counselors on their core counseling competencies, (5) Provide feedback to 
counselors. The SPs and study supervisors reported that the checklist was easy to use when rating the counselors. 
Counselors reported that the SP encounters were realistic and helpful for practicing their clinical work and for 
building self-efficacy for working with real patients. The labor costs of developing two SP cases and training two SP 
actors was approximately $1,475. The per-session labor cost of conducting a 1-hour counseling session between 
one SP and one counselor was approximately $314. 

Conclusions: Using SPs to train telephone counselors working on a clinical trial was feasible and offered training 
benefits beyond those provided by didactic instruction and role plays. Our research group is now routinely using 
SPs for the training of incoming telephone counselors. 

Keywords: Standardized patient, Clinical trial, Feasibility, Implementation, Behavior change 



Background 

Standardized Patients (SPs) are actors trained to consist- 
ently portray health care patients based on well-developed 
clinical scenarios [1,2]. SPs are often used in the training 
and assessment of clinicians and have been used during 
the implementation of clinical trials to improve study 
procedures prior to working with real patients [3-5]. 
This paper describes the methods and costs associated 
with using SPs to evaluate the skills of telephone counselors 
working on a study that evaluated a telephone smoking ces- 
sation program for smokers using Department of Veterans 
Affairs (VA) mental health clinics. The goal of the parent 
study was to evaluate the effectiveness of an intensive 
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telephone smoking cessation counseling program for VA 
mental health patients that included behavioral change 
counseling and problem solving therapy approaches [6-8]. 

Methods 

The parent study received approval from the Institutional 
Review Boards and Research and Development Commit- 
tees at participating VA facilities. The study employed two 
counselors who underwent 15 hours of initial protocol 
training using didactic instruction and role-plays. After 
completing lectures and role-plays, the counselors com- 
pleted two SP encounters. Developing and conducting the 
SP encounters required five main steps (Table 1). 

Step 1 - Develop the SP cases 

Each counselor was to complete two SP encounters, so 
we developed two SP cases using our affiliation with 
New York University School of Medicine (NYUSoM). 
NYUSoM has a bank of SP cases, about a third of which 
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Table 1 Steps required to conduct an SP exercise 



Step 


Resources required 


1 Develop the SP case 


• Case materia s from a loca school of 




medicine or professional accreditation 




bodies (e.g., AAMC) 




• Expert input for adapting or developing 




a case 


2. Identify, se ect and train 


• List/pool of potentia actors 


SP actors 


• 2-3 hour training session 






• Actor payment (local rates) 


3. Conduct the SP encounter 


• Scheduled encounters between SPs 




and trainees 




• Audio or video recorder 




■ Actor payment 




• Debriefing session (optiona) 


4. Rate trainee performance 


• Standardized trainee assessment 




form and anchors 




• Actor payment (if SPs will rate trainees) 


5. Provide feedback to trainee 


• One-on-one or group feedback session 




with trainees 




• Feedback can come from SP and/or 




supervisor 




■ Actor payment (if SPs will debrief with 




trainee) 



SP = standardized patient. 

AAMC = American Association of Medical Colleges. 



focus on promoting behavior change. From NYUSoM's 
case bank we selected two cases to adapt for our study. 
The first was a smoking cessation case, to which we 
incorporated a history of depression. The second was a 
post-traumatic stress disorder (PTSD) case, to which we 
incorporated a history of smoking. To further adapt the 
cases for a VA population, we made one case a Caucasian 
male in his 20s who served in Iraq and the other case an 
African American male in his 60s who served in Vietnam. 
Table 2 provides an overview of case #2. The study's 
clinical supervisor, a VA clinical psychologist, wrote an 
extensive smoking, military, and mental health history 
for each case, and a careful description of how to respond 
to counseling questions, counselor tone and demeanor, 
and the counselor's proposed cessation treatment plans. 

Step 2 - Identify, select and train standardized 
patient actors 

Because our study population was Veterans with mental 
health disorders, playing a study participant would be 
emotionally demanding and challenging [9]. Therefore, 
we employed professional SP actors to accurately convey 
emotions and behavioral symptoms common in mental 
health patients. To locate SPs, we used NYUSoM's pool of 
experienced SPs. Following recommendations in the lit- 
erature for SP recruitment [2], we contacted actors who 
had experience playing the two cases adapted for the study 



and who matched the gender, age and racial demographics 
of the cases. After hiring two SPs, we emailed the assigned 
cases to each SP. The SPs reviewed their case prior to an 
in-person group training session with the study's project 
director (ESR) and one of the study's co-investigators 
(CG) who has SP training experience. The in-person train- 
ing lasted 3 hours and provided the SPs with background 
information about the study and its counseling protocols 
(including a discussion of how to respond when the study 
counselors do and do not use appropriate counseling 
techniques), a detailed overview of each case, the oppor- 
tunity to ask questions, and training in how to assess 
the study's counselors using a checklist developed for 
the study (see Step 4 for checklist details). 

Step 3 - Conduct the SP encounters 

The study's project director scheduled telephone appoint- 
ments for the counselors and SPs, who were located in sep- 
arate rooms in the hospital. The counselors called the SPs 
and completed a first telephone counseling session using 
the methods they would use with study participants. We 
audio-taped the sessions using a digital recorder connected 
to each counselor's phone, and the study co-investigator 
leading the SP encounters listed-in "live" on each call. 

Step 4 - Rate trainee performance 

SPs and trainers can consistently and accurately rate 
trainee performance with well-developed assessment 
tools [10,11]. Therefore, immediately after the counseling 
sessions, the SPs and the co-investigator present at the SP 
encounters rated the counselors using a 35-item checklist 
created for the study (Figure 1). To create the checklist we 
combined the validated Behavioral Change Counseling 
Index [12] with items specific to our study's counseling 
protocol. Using Likert-type scales, the SPs and co- 
investigator scored the presence and quality of smoking 
cessation counseling behaviors outlined in the study's 
counseling manual and counseling behaviors in accord- 
ance with behavioral change counseling techniques [7]. At 
a later date the study's clinical supervisor listened to the 
session audiotapes and completed the checklist for each 
session. 

Step 5 - Provide feedback to trainee 

Immediately after the SP encounters, the co-investigator 
present at the encounters, the SPs, and the counselors met 
for a 30-minute debriefing session. The co-investigator and 
SPs provided counselors with feedback on their perform- 
ance. The counselors discussed how they felt during and 
after the encounters, including their opinion on whether 
the encounters felt real to them and how the cases could be 
improved. 

At a later date the clinical supervisor provided one- 
on-one feedback to each counselor using the checklists 
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Table 2 SP case #1 



Case element 



Example 



SP demographics 
SP demeanor 

SP clinical diagnosis, history, and symptoms as necessary 



Other SP psychosocial history 



Instructions on how to respond to intervention competencies 
demonstrated by trainee during the SP encounter 



60 year-old African American male. Married for 28 years. Divorced for 7 years. 
Two children (ages 26 and 30). Unemployed for 8 years. Receives all health 
care from the VA. 

Friendly and open to cessation counseling but reserved when speaking about 
his military history. 

Smoking: Began smoking at age 19. Smokes 10-15 cigarettes per day. Used to 
smoke up to 2 packs per day. Decreased his smoking about 10 years ago when 
cigarettes began to cost more. Likes to smoke for relaxation. Has tried to quit a 
few times in the past. Most quits were "cold turkey." Has tried the nicotine gum. 
Has never tried counseling to quit smoking. If asked on a scale of 0-10 his 
motivation to quit smoking, patient will report a 7. Motivated to quit smoking 
because it is too expensive and because he knows it is bad for his health. If 
asked on a scale of 0-10 his confidence in being able to quit, patient reports 
a 5. He is not very confident because of his struggles with stress and his 
previous relapses. He is unsure if he will be able to cope with his PTSD and 
stress without cigarettes. 

Mental health: PTSD linked to military service during Vietnam war. Suffers from 
insomnia, flashbacks of his best friend being killed in front of him, and heightened 
startle reflex to loud noises. History of alcohol abuse but sober for 3 years. 
Recovered from alcohol abuse using AA (which he still attends) and religion 
(Catholic). Very proud of his sobriety. 

Wife left him 7 years ago due to his drinking. Does not have great relationship 
with his adult children and would like to be closer to them. They live in NYC 
but he rarely sees them. Lost his last job (NYC bus driver) due to his drinking. 
He has many friends through AA and VA programs. Most of his friends smoke 
and may not support his quitting. 

Begins call in "contemplation" stage of change: thinking of quitting but not yet 
ready to set a quit date or commit to quitting. If the counselor uses appropriate 
counseling techniques (see protocol overview), the patient can commit to quitting. 
If the counselor does not use appropriate counseling techniques, the patient will 
remain unsure of his motivation to quitting. 



completed by the supervisor and SPs. The clinical super- 
visor also reviewed the checklists to certify each counselor 
as ready to work with real patients. To be ready to work 
with real patients counselors needed the following score 
patterns: 1) On the Counseling Protocol scale counselors 
needed a rating of "Did Well" on the majority of items 
and no ratings of "Did Not Do;" 2) On the BECCI coun- 
selors needed a rating of "A Good Deal" or better on the 
majority of items and no ratings of "Minimally" or lower 
unless an item was not applicable; and 3) On the Overall 
Ratings scale counselors needed a rating of "Very Much" 
on the majority of items and no ratings of "Not at all." 

Assessing costs 

We calculated the labor costs associated with developing 
and conducting the SP exercises. We structured these 
costs into two categories: development costs (steps 1 
and 2) and per-session costs (steps 3-5). To calculate 
the development costs we estimated the number of 
hours it took for personnel to develop two SP cases 
and identify and train two SP actors. We multiplied 
these hours by personnel hourly rates in 2009 dollars 
(the year in which we conducted the exercises) and 
added NYUSoM's fringe benefit cost. To these study 



personnel costs we added the costs of the two SPs, 
which were calculated as the SPs' hourly rate multiplied 
the number of hours each SP spent in training. 

For per-session costs we calculated labor costs asso- 
ciated with a 1-hour counseling session between one 
counselor and one SP. We provide estimates for one 
hour of counseling, because this was the recommended 
length of time it would take to complete an intensive 
first counseling session with our difficult study population. 
As with the development costs, we multiplied the num- 
bers of hours each study staff member worked on an SP 
session by his/her hourly rate and added NYUSoM's fringe 
benefit cost. SP costs were calculated as the SP's hourly 
rate multiplied the number of hours an SP spent in coun- 
seling and performing trainee rating and feedback. 

Findings 

During the debriefing session the SPs and supervisors 
reported that the checklist was easy to use when rating 
the counselors' competencies, and the counselors reported 
that they found the ratings to be helpful in understanding 
how they performed during the SP exercises. Both coun- 
selors reported that the SP exercises were helpful for prac- 
ticing their clinical work and for practicing the logistics of 
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SP: 




Counselor: 




Date: 




Case: 





COUNSELING PROTOCOL 





Did NOT Do 


Did but Could 
Have Done Better 


Did 
Well 


Asked for my permission to talk about quitting 


□ 


□ 


□ 


Obtained my full history of smoking 


a 


D 


□ 


Assessed my stage of quitting 


a 


□ 


a 


Asked how important it is for me to quit 


□ 


D 


□ 


Explored my confidence in my ability to quit 


a 


□ 


a 


Explored links between my smoking and mental health 


a 


□ 


□ 


Elicited my pros and cons of smoking 


a 


a 


a 


Elicited my triggers for smoking 


Q 


□ 


□ 


Comments: 


Was accepting/affirming (NOT judgmental) 




Q 


□ 


Encouraged mc to talk about my smoking 










D 


Helped mc ID strategics for coping with triggers 










1 


Helped mc come up with a quit plan 












Helped mc set goals related to smoking 










1 


Used refraining to sec things in a new way 












Facilitated behavior change (did NOT direct) 










□ 


Comments: 


Provided info in understandable ways 








Answered questions 








Checked to make sure 1 understood information 








Comments: 



BEHAVIOR CHANGE COUNSELING INDEX (BECCI) 









To 


A 


A 






Not At 


Mini- 


Some 


Good 


Great 




To what degree were each of the following actions carried out? 


All 


mally 


Extent 


Deal 


Extent 


N/A 


Invited mc to talk about behavior change 


0 




2 


3 


4 


N/A 


Demonstrated sensitivity to talking about other issues 


0 




2 


3 


4 


NA 


Encouraged mc to talk about current behavior/status quo 


0 




2 


3 


4 


N.A 


Encouraged mc to talk about change 


0 




2 


3 


4 


NA 


Asked (open-ended) 7s to find out what I think and feel 


0 




2 


3 


4 


N.A 


Used cmpathic listening statements (reflection) 


0 




2 


3 


4 


N/A 


Used summaries to bring together what I said 


0 




2 


3 


4 


N.A 


Acknowledged challenges I face in changing behavior 


0 




2 


3 


4 


N/A 


When provided info, was sensitive to my concerns and 


0 




2 


3 


4 


N/A 


understanding 














Actively conveyed respect for my choices about behavior change 


0 




2 


3 


4 


N/A 


We exchanged ideas about how I could change my current 


0 




2 


3 


4 


N/A 


behavior 















OVERALL RATINGS 





Not At 


Only a 


Some- 


Very 




Please describe the degree to which the counselor.... 


All 


Little 


what 


Much 


Comments 


Fully explored my experience of smoking/trying to quit 


0 


1 


2 


3 




Took a personal interest in me; treated me as a person 


0 


1 


2 


3 




Actively engaged me 


0 


1 


2 


3 




Demonstrated concern, intention to help 


0 


1 


2 


3 




Was empathic/understanding 


0 


1 


2 


3 




Motivated me to change my smoking behavior 


0 


1 


2 


3 




Additional Comments/Things of Note: 



Figure 1 Counseling assessment form (SP Version). 



working within a research setting. Both counselors also re- 
ported that working with the SPs allowed them to release 
some nervousness and build self-confidence in working 
with real patients. The counselors reported that they 
found the SP encounters to be realistic but reported two 
major weaknesses in the SPs' abilities to portray VA men- 
tal health patients. One SP was not familiar with common 
military terminology of his combat era, and a second SP 
was not able to let the counselor know if he was taking 
any psychiatric medications. These lapses in knowledge 
were salient to the counselors and affected their ability to 
perceive the encounters as real. 

Based on the success of the SP exercises with our first 
two counselors, we used SPs to train three subsequent 



study counselors after making some changes to our 
methods. We modified the cases to include more infor- 
mation about each SP's military history and ongoing 
mental health treatment. In addition, to more closely 
resemble the procedures that would be used with real 
patients, the counselors called the SPs at their homes, 
we did not have a study team member listening-in on 
each call, and we did not have the counselors meet 
their SPs for an in-person debriefing session. Instead, 
the SPs provided feedback over the phone immediately 
after their counseling call. 

Table 3 provides a summary of the labor costs associ- 
ated with developing and executing the SP exercises 
using our modified session methods described in the 
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Table 3 Estimated development and per-session costs of executing the SP training exercise 



Development costs 


Tasks 


Personnel/SP 


Hours 


Hourly Hourly 


Fringe 


Total 










rate 


total 




cost 


Step 1. Develop 2 SP cases 


Identify appropriate case from 


Clinical Psychologist 


5 


$38 


$190 


35% 


$257 




NYUSoM case bank. Adapt the 
case for current study. 


Co-investigator with SP expertise 


5 


$38 


$190 
Step 1 Total 


35% 


$257 
$515 


Step 2. Identify, select and 


Identify appropriate actors from 


Project Director 


5 


$29 


$145 


35% 


$196 


Train 2 SP actors 


NYUbOIVl jr pools. Lontact jrS 

and invite to participate. Develop 


Co-investigator with SP expertise 


10 


$38 


$380 


35% 


$515 




and conduct 3-hour training 


cpi 

jr 1 


C 

J 


$25 


$125 


n/a 


i 1 is. 

5 I ZD 




llltrfcrllliy WIU1 Z JrS. Z jri IcdU 


SP2 




$25 


$125 


n/a 


$125 




and study case on their own. 


5 




Arrange for SP payments. 








Step 2 total 




$961 










Total development costs 


$1,475 


Per session costs 3 


Tasks 


Personnel/SP 


Hours 


Hourly Hourly 


Fringe 


Total 










rate 


total 




cost 


Step 3. Conduct SP encounter 


Schedule counseling appointment. 


Project Director 


0.5 


$29 


$15 


35% 


$20 




Counselor calls SP and provides 
1 hour of counseling per protocol. 


Counselor 


1.25 


$25 


$31 


35% 


$42 




Counselor completes call 


SP 


1 


$25 


$25 


n/a 


$25 




documentation. 








Step 3 total 




$87 


Step 4. Rate trainee performance 


Supervisor listens to audiotape. 


Clinical Psychologist 


1.5 


$38 


$57 


35% 


$77 




Supervisor and SP complete 
rating form. 


SP 


0.5 


$25 


$13 
Step 4 total 


n/a 


$13 
$90 


Step 5. Provide feedback to 


SP provides feedback immediately 


Clinical Psychologist 


1 


$55 


$55 


35% 


$74 


trainee 


after counseling call. Supervisor 
provides feedback in 


Counselor 


1.5 


$25 


$38 


35% 


$51 




one-on-one meeting with 


SP 


0.5 


$25 


$13 


n/a 


$13 




counselor. 








Step 5 total 




$138 










Total per session cost 


$314 



SP = Standardized Patient, NYUSoM = New York University School of Medicine. 
a Per-session costs are for 1 hour of counseling between 1 SP and 1 counselor. 



previous paragraph. The cost of developing the SP 
cases and training SP actors was approximately $1,475. 
The per-session cost of conducting a 1-hour counsel- 
ing session between one SP and one counselor was 
approximately $314. 

Discussion 

The literature supports the use of SPs to train and assess 
clinicians [1,5]. Our study supervisors found the use of 
SPs to be effective in assessing counselor readiness to 
work with real patients, and the counselors reported that 
they found the SP exercises to be realistic and helpful 
for their training. Labor costs associated with the use of 
SPs for counselor training were modest and comparable 
to costs found in prior research piloting the use of SPs 
to train substance abuse counselors [9]. 

Our methods have several limitations. First, we told 
the counselors that they would be treating SPs, which 
could have affected the way in which they provided 
treatment [13]. Second, we only tested the counselors on 
a single first call with each SP rather than have them 
complete a full course of counseling with an SP. Third, 



because we did not assess the counselors' competencies 
with our checklist before the SP encounters, we are not 
able to determine if the counselors' skills improved as a 
result of the SP exercise. Fourth, the BECCI portion of 
our checklist has been validated in prior research, but 
the two sections of the checklist created for this study 
were not validated. Lastly, generalization of our findings 
may be limited. Our study took place in a city with a 
relatively high cost of living. Therefore, labor costs may 
be significantly higher than if the study were conducted 
in a different city. In addition, the per-session costs ne- 
cessarily depend on the length of a counseling session. 
Costs would be lower to train counselors for briefer 
interventions. 

In summary, the purpose of this paper was to describe 
the application of the standardized patient training 
methodology to the assessment of counselors working 
on a clinical trial. The application of the methodology 
was feasible, inexpensive and offered training benefits 
beyond those provided by didactic instruction and role 
plays. Our research group is now routinely using SPs 
for the training of incoming telephone counselors. 
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